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Abstract — This work addresses private communication 
with distributed systems in mind. We consider how to best 
use secret key resources and communication to transmit 
signals across a system so that an eavesdropper is least 
capable to act on the signals. One of the key assumptions 
is that the private signals are pubUcly available with a 
delay — in this case a delay of one. We find that even if 
the source signal (information source) is memoryless, the 
design and performance of the optimal system has a strong 
dependence on which signals are assumed to be available 
to the eavesdropper with delay. 

Specifically, we consider a distributed system with two 
components where information is known to only one com- 
ponent and communication resources are limited. Instead 
of measuring secrecy by "equivocation," we define a value 
function for the system, based on the actions of the 
system and the adversary, and characterize the optimal 
performance of the system, as measured by the average 
value obtained against the worst adversary. The resulting 
optimal rate-payoff region is expressed with information 
theoretic inequalities, and the optimal communication 
methods are not standard source coding techniques but 
instead are methods that stem from synthesizing a mem- 
oryless channel. 

I. Introduction 

Consider a situation where an adversary attempts to 
disrupt a distributed system. The adversary may be 
attempting to jam communications, destabilize a power 
grid, or counter a miliary attack. We investigate the 
nature of the communication used to control the system. 
How should the system use secret key resources to 
establish coordinated behavior? How should the system 
disseminate information about availability of power in a 
power grid, frequency channels or network routes used 
for communication, or military strategy adjustments? 
This work establishes a new approach for defining se- 
crecy in distributed systems, robust against an attacker 
who can causally observe the behavior of the system and 
apply statistical attacks to make use of the intercepted 
communication signals. We find that the optimal way to 
communicate with limited encryption capability (secret 



key) is to reveal a distorted version of the information 
in the clear and use the encryption resources to hide the 
most important aspects of the information. 

A large body of theoretical research provides a foun- 
dation for understanding privacy in communication. The 
bulk of this research focusses on creating private chan- 
nels from limited physical resources. A formal informa- 
tion theoretic study began with Shannon's 1949 paper, 
"Communication Theory of Secrecy Systems" [1], where 
he established that the Vernam cipher using a one-time 
pad was necessary and sufficient for perfect secrecy. 
Thus, we understand how to use a random secret key 
to create a private communication channel. But how 
do we create and distribute this secret key? Diffie and 
Hellman gave birth to modern cryptographic methods in 
1976 |2| by showing how to use "trapdoor functions" to 
enable key distribution using communication over public 
channels. The resulting keys are protected by computa- 
tional complexity but are not theoretically proven to be 
secret. Nevertheless, they have withstood tests of time 
and enabled the broad commercial use of encryption. 
Some different approaches for distributing secret keys 
use quantum channels f3l or observations of correlated 
variables f5\. 

Another method for creating private communication 
channels has had extensive recent interest. Without the 
use of a shared secret key, channel noise can be used 
to hide a message if the noise is different for the 
eavesdropper than it is for the intended receiver. This 
so-called "physical layer security" began with Wyner's 
famous wiretap channel [6 |. Now a variety of situations 
are studied, with multiple sources of information, fading, 
or multiple parties involved (see for example [7 1). 

In this work we do not worry about establishing a 
secret key or creating a private channel. We assume that 
such a resource is already available, and we investigate 
how to best use it. Just as channel coding and source 
coding share a complementary relationship — channel 
capacity characterizes the creation of communication 
resources from physical constraints, and rate-distortion 



theory establishes the best use of such resources — we see 
this work as a complementary development to the main 
body of information theoretic secrecy research, which 
is typically concerned with creating private communi- 
cation resources. We show how to use those resources 
to achieve optimal performance in a particular broad 
setting. 

In our framework, the transmitter and intended re- 
ceiver of communication are part of a distributed system. 
A sequence of information is known at the transmit node, 
but the objective may not be to send that information 
to the receiver node verbatim. Instead, the receiver will 
be producing a sequence of actions important to the 
distributed system that should be somehow correlated 
with the information sequence. This is captured by a 
payoff function, the average value of which the com- 
munication system is designed to maximize. So far this 
falls exactly in the category of rate-distortion theory. 
However, the catch is that an adversary, who is also 
observing the communication over the public channel, is 
able to perform actions that will also affect the payoff. 

This view of a secrecy system puts the adversary and 
the communication system in a zero-sum game against 
each other Indeed, when talking about security, a game 
theoretic formulation seems appropriate. The optimal 
communication system in our framework is designed to 
maximize the average payoff against the most clever ad- 
versary. Measurements of the uncertainty of the informa- 
tion with respect to an adversary, such as "equivocation," 
do not indicate how useful the intercepted information is. 
We work directly with an operational quantity — average 
payoff. 

The main result, given in Section JIIJ corresponds to 
an encoding scheme that is quite different from optimal 
encoding techniques commonly used for source coding. 
A reasonable question to explore is, "which parts of 
problem statement are pivotal for these results?" We 
find, for example, that the information we assume is 
available to the adversary is of primary importance. 
In our framework, we assume the adversary not only 
intercepts the public communication but also observes 
the past information and actions of the system. If we 
weaken the adversary by reducing the information that 
is available, then the results change substantially. This is 
explored in Section IVIII 

On the other hand, some details of the problem state- 
ment are not crucial. Although our problem formulation 
defines the payoff of the system to be a function of three 
variables — the information, the action of the intended 
receiver, and the action of the adversary — a small mod- 
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Fig. 1. Distributed System with Adversary. The information sequence 
X" is i.i.d according to po{x). Node A and Node B are designed so 
that Node B can produce a sequence of actions y" which depend 
on X". The resources available to them are communication over a 
public channel at rate R bits per action and secret key (independent 
of X") at rate Rq bits per action. An adversary taps into the message 
sent over the public channel and observes the actions of the system 
and information sequence causally. The adversary attacks the system 
by minimizing a value function n{x, y, z), and the system is designed 
to maximize the worst case average value obtained. 



ification would be to have two objectives to separately 
handle the behavior of the system and the adversary. 
The first objective could be a distortion constraint at the 
intended receiver, and the second objective could be to 
force a level of distortion on the adversary. In this case, 
the two distortion functions, each of only two variables, 
take the place of the payoff function. It may seem that 
this special case would allow for a simplification of the 
main result given in Section 13.11 allowing for a much 
more basic optimal communication scheme. Surprising, 
this does not seem to be true (see Section IVI-BI ). 



II. Problem Statement 

In Figure [T] a distributed system is represented by 
two components. Node A and Node B. A third node. 
Node C, represents a hypothetical adversary who attacks 
the system. An i.i.d. source of information {Xj}^^ is 
known to Node A. The distribution of each element of 
the sequence is given by the probability mass function 
Po{x). 

Node B and Node C take actions in the system. A 
function 7r{x, y, z) represents the value obtained by the 
system during each time epoch when the information 
symbol Xi is x, the action Yi by the cooperating agent 
at Node B is y, and the attack Zi by the adversary is z. 
The system, which operates over blocks of length n, is 



designed to maximize the average value 

— 1 " 

n = ^-y^7r{Xi,Yi,Z.i), 

i=l 

while the adversary tries to minimize 11. One interpreta- 
tion of IT is as the payoff function of a zero-sum game 
between the distributed system and the adversary. 

The communication in the system is defined as fol- 
lows. Node A sends a message to help Node B coordi- 
nate actions with the source sequence {Xj}^^, but the 
message is available to both agents, Node B and Node 
C. However, a secret key (independent of the source 
sequence) is known only to Nodes A and B, which 
can be used to establish secrecy and coordination in the 
system. For example, if the secret key is large enough, 
the communication can be fully encrypted so that only 
Node B can make use of it. 

An {Ro,R,n) coordination scheme consists of an 
encoder and a decoder utilizing a secret key rate of 
Rq bits per symbol and a description rate of R bits 
per symbol. The encoder at Node A transmits an nR- 
bit message J e [2"^] based on the source realization 
X" and an nRo-hit secret key K € [2"^"] which is 
independent of the source. The encoding at Node A can 
be designed to use randomization; thus, it is described 
by a conditional probabiUty function p{j\x'", k). 

The attack by the adversary (Node C) on the dis- 
tributed system (Nodes A and B) occurs interactively. 
Nodes B and C first receive the communication produced 
by Node A. Then they each produce one action, Yi 
and Zi. This constitutes the first instance of the game, 
after which both nodes are aware of the action taken 
by the other node and the first symbol of the source 
realization Xi. At this point they each choose a second 
action, I2 and Z2, and proceed in a similar manner. 
For each iteration of the game, the decoder at Node 
B generates an action based on the message J, the 
secret key K, and the past actions X*^^, y*^^, and Z'^^. 
This decoder is described by a set of conditional proba- 
bility distributions {p{yi\j,k,x^'~^,y^^^, z'^~^)}f^^. The 
adversary (Node C) also generates actions in a similar 
way as the coordinating agent (Node B), except that he 
doesn't have access to the secret key. His actions are 
described by a set of conditional probability distributions 
x*~^, 2*"^)}"^-^. We consider the strategy 
of the eavesdropper that inflicts the most damage on the 
system. An {RQ,R,n) coordination scheme is evaluated 
by the expected average payoff it assures against the 
worst-case adversary. 



To summarize: 

• Source: {X^} i.i.d. po{x) 

• {Ro,R,n) Coordination Scheme: 

- Key: K Unif[2"-^"] independent of source 

- Message: J G [2"-^] 

- Encoder (Node A): 

Ca ^ p(j>",A:) 

- Decoder (Node B): 

Cb = {pim\j,k,x'-\y'-\z'-'m^, 

• Adversary 

- Strategy (Node C): 

Cc = {p{zi\j,x'-\f-\z'-')}U 

• Joint Distribution: product of all of the above. 

• Average Value: 

- 1 " 

Up,{Ca,Cb,Cc) = E-y^7T{Xi,Yi,Zi), 

i=l 

• Robust achievable value: 

Ilpg{Ro, R) = sup min 11 

{n,CA,CB} 

Note: A system may not be aware of an adversary 
or the actions an adversary has taken. There may be 
multiple adversaries attacking a system. This problem 
statement defines a decoder at Node B that responds to a 
single adversary. However, we will find that the optimal 
max-min codec does not take the actions of adversary 
into account, nor does it use the causal source informa- 
tion. Decoders at Node B of the form Cb = {piVilJ, k)} 
achieve optimality. Therefore, these results are more 
widely applicable than the specific setting described. 
They may also apply to situations where an adversary 
is not easily detected or multiple adversaries exist. 

On the other hand, the causal information available to 
the adversary is crucial for these results. If the adversary 
had less information available, the system can achieve 
the same value using less resources. In the extreme case, 
if the adversary has no causal information of the actions 
at Node B or the information at Node A, then it is easy to 
transmit a message that is useless to the adversary. Any 
small rate of secret key is as good as perfect secrecy in 
the sense that the adversary cannot mount an attack based 
on the intercepted message. This is discussed further in 
Section |VlIl 



III. Main Result 



Theorem 3.1: 

np(,(-Ro,-R) = 

where 

Vp,{Ro,R) ^ 



max minF, 7r{X,Y, z{U)), 

p{y,u,v\x)£'P z{u) 



p{y\u,v,x) = p{y\u,v), 

Ro > I{X,Y;V\U), 
R > I{X;U,V). 



IV. Converse 

Proof: In this section we prove an upper bound on 
Ilpg{Ro, R). For any coordination scheme satisfying the 
rate constraints Rq and R, we identify random variables 
X, Y, U, and V such that p{y, u,v\x) G Vpg^Ro, R) and 

minlT = minE 7r(X,y,2(C/). 

Cc z(u) 

We first identify the random variables for the converse. 
Let Q be a random variable uniformly distributed on 
the set [n] and independent of (X", y", Z", J, if). We 
will use Q as a random index for sequences, where Xq 
is a function of the sequence X" and the variable Q 
that selects the Qth element of the sequence. Notice 
that for an i.i.d. sequence like X"-, the random index Q 
is independent of Xq. 

Identification of variables: 



nRo > H{K) 

> H{K\J) 

> I{X'',Y'',Z'';K\J) 

n 

= nI{XQ,YQ,ZQ-K\J,X'^-\Y'^-\Z^-\Q) 
= nI{XQ,YQ;K\J,X'^-\Y'^~\Z<^-\Q) 
= nI{X,Y;V\U). 

nR > H{J) 

> H{.J\K) 

> /(X";J|K) 
= IiX^;J,K) 

n 

= Yl{Xk;J,K,X^-') 

9=1 

n 

= Yl{Xk;J,K,X'~'\Y'-\z'-') 

9=1 

= nI{XQ;J,K,X'^-\Y'^-\z'^-\Q) 
= nI{X;U,V). 

Consequently, the variables U, V, and Y are 
conditionally distributed according to p{y,u,v\x) G 
Vpg{Ro, R). Now consider that 



» X = Xq 




' Y = Yq 


minll = 

Cc 


• Z = Zq 




. V = K 




. u = {j,xQ-\yQ-\zQ-\q} 




Important properties: 




• Independence 





mm 



1 " 

B-y^7T{Xi,Y^,Zi) 



Y, 



Zi, 



Xq ± Q, 

• Markovity 

(Xr,Z,) - {J,K,X'-\Y'-\Z'-') 
{Xl\Y,,K) - {J,X'-\Y^-\Z'-') 
(X,) - {J,K,X'-') - {Y\Z'). 

We start by noticing that X is distributed according to 
Pq and X — ([/, V) — Y form a Markov chain, according 
to the definitions and properties above. 

Next, we show the inequalities that involve Rq and R. 



{p{z,\j,x--\y^-\z--~^)yU n 

min EE K^q, Fq, Zq)|Q] 

1,29- 1,(7) 

min E 7r(X, Y, Z) 

p{z\u) 

minE7r(X,y,z(C/)). 

z{u) 



V. Sketch OF AcHiEVABiLiTY 

Proof: Here we design a system that guarantees 
an average expected reward approaching the value of 
IIpo (i?o 1 R) given in Theorem 13.11 This is done using 
the notions of empirical coordination and strong coordi- 
nation discussed in fF]. 

Begin with the optimal conditional distribution 
p{y,u,v\x) G Vpg{Ro, R). The main idea is to first 
specify a U"' sequence that is empirically coordinated 
with X", which is to say that it is jointly typical 
with high probability, using a communication rate of 



roughly I{X; U) bits per source symbol. Then produce 
a sequence that is strongly coordinated with X", 
conditioned on [/", which is to say that X" and y" 
appeal- to be memoryless, even with full knowledge 
of the codebook used for coordination. The variable 
V is an auxiliary variable that only has meaning in 
the process of achieving strong coordination. The rates 
needed for strong coordination over a public channel 
are I{X,Y;V\U) bits of secret key per symbol and 
I{X;V\U) bits of communication per symbol, with the 
condition that X — {U, V) — Y form a Markov chain. 
These rate requirements are touched on in |0. 

After encoding, the adversary knows the sequence 
The other sequences in the system, X"- and Y"-, 
are correlated with [/", but otherwise appear to be 
nearly memoryless (in total variation), even conditioned 
on everything known by the adversary. Therefore, the 
best strategy for the adversary to minimize the average 
value to the system will not be substantially better 
than choosing the best strategy z{u) that minimizes 
E 7r(X,Y, z{U)) and applying this strategy during each 
iteration. ■ 



VI. Special Cases 



A. Lossless 



In some cases of vr, the description of Ilpg{RQ, R) 
simplifies. A particular important case is the lossless 
setting, where y" is required to be equal to X" with 
high probability^] This is worth considering because it 
resembles the familiar setting usually considered for 
information theoretic secrecy, where the information 
source sequence X" must be recovered by the intended 
receiver. 

In the lossless case, vr can be though of as a distortion 
function that is being inflicted on an eavesdropper. The 
system is designed to maximize the distortion. 

The following corollary is found in ifTOl and can be 
reduced to a linear program. 

Corollary 6.1: For the lossless case, where y" must 
equal X" with high probability, the robust average value 
is 



Ilpg{Ro,R) = max minE 7r(X, z(C/)), 

p(?i|x)gP z{u) 



where 



p{u\x) : 

Vp,XRo,R) = { Ro > H{X\U), 

R > H{X). 



B. No interaction 

We can imagine some cases where the intended re- 
ceiver and the adversary each attempt to reconstruct the 
source X" with low distortion, and they are each not 
concerned with the reconstruction of the other. This situ- 
ation can be expressed with two separate value functions 
7ri(x,y) and 7r2(x,2;). We might ask for two separate 
constraints to be satisfied or simply construct a value 
function 7r(x, y, z) that is the product or sum of these 
two separate components. 

By removing the direct interaction between the in- 
tended receiver and the adversary, it might seem that a 
simpler encoding scheme, not involving strong coordi- 
nation, is optimal. Unfortunately, that is not the case. If 
the distortion constraint of the intended receiver were 
achieved in a careless way, without using strong coordi- 
nation, then the adversary would be able to infer extra 
information about X^ indirectly through observation of 
the past actions of the intended receiver. This would help 
the adversary to reduce its own distortion. 

VII. Limited Adversary 

The result expressed in Theorem 13.11 specifies a 
guaranteed average value against an adversary who has 
causal information of all actions and information in the 
system. The results change significantly if the adversary 
is limited in the information available. Here we give three 
results related to the main result in Theorem 13.11 with 
proofs omitted. 

Theorem 7.1: Consider an adversary who only has 
access to the message J and the past actions at Node B, 
but not the past information symbols. Thus, the strategies 
of the adversary are defined by {p{zi\j,y'^~^ ^ z^~^)}'^^^. 



npo(i?o, R) 
where 

'PpoiRo^R) ' 



max minE 7r(X, y, 2:([/)), 

p(y,u,v\x)(iV z{u) 



p{y,u,v\x) 
p{y\u,v,x) 
Ro 



> 



p{y\u,v), 
I{Y-V\U), 



R > I{X;U,V). 



Theorem 7.2: Consider an adversary who only has ac- 
cess to the message J and the past information symbols, 
but not the past actions at Node B. Thus, the strategies 
of the adversary are defined by {p{zi\j, x^~^, 



'This would be the case if Tv{x,y,z) took on veiy large negative 
values for x y. 



Ilp^{Ro,R) = max minE 7r(X, y, 

p{y,u\x)£'P z{u) 



where 

!p{y,u\x) : 1 

Ro > IiX,Y\U), \. 

R > I{X;U,Y). J 

Theorem 7.3: Consider an adversary who only has 
access to the message J. Thus, the strategies of the 
adversary are defined by {^(^^ilj, 

np(,(iio)-R) = ™ax minE 7r(X, y, z), 

p{y\x)eV 2 

where 

{p{y\x) : \ 
i?o > 0, \. 
R > I{X;Y). J 

The third setting, in which the adversary has no causal 
information, is the approach taken by Yamamoto in [111 . 
We see from Theorem 17.31 that there is no non-trivial 
lower bound on the rate of secret key needed. In Theorem 
3 of |TT| a lower bound is provided, but after careful 
consideration and with the proper choice of auxiliary 
random variables, that bound can be shown to be zero. 

In other work by Yamamoto |[T2l . Rq is taken to 
be exactly zero. The results do not have the form of 
Theorem 17.31 Instead they coincide with Theorems 13.11 
17.11 and 17.21 with Rq = 0. This illustrates a discontinuity 
of Up^XRo,R) at Ro = 0. 

VIII. Summary 

We've identified the optimal use of communication 
and secret key resources for a distributed system to 
compete in what amounts to a zero-sum game with an 
adversary. This brings new insight into the theoretical 
limits of secrecy systems. The results found here are not 
directly related to information measures such as "equiv- 
ocation," which are commonly used in the literature. 

Perhaps the most surprising, yet subtle consequence 
of the main result is that the combined resources of 
public communication and secret key of equal rates is 
strictly superior to a single private channel resource of 
the same rate. Furthermore, a secret key rate in excess 
of the public communication rate can actually be useful. 
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